Change in Notation. e (i) (n) Note that this is inconsistent with the notation used earlier, c k+1x(n k) e (i) (n) x(n i) ˆx(n i) M

Transcription

1 Overview of Linear Prediction Terms and definitions Nonstationary case Stationary case Forward linear prediction Backward linear prediction Stationary processes Exchange matrices Examples Properties Introduction Important for more applications than just prediction Prominent role in spectral estimation, Kalman filtering, fast algorithms, etc Prediction is equivalent to whitening! (more later) Clearly many practical applications as well J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 1 J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 2 Nonstationary Problem Definition Given a segment of a signal {x(n),x(n 1),,x(n M)} of a stochastic process estimate x(n i) for i M using the remaining portion of the signal where c i (n) 1 ˆx(n i) c k(n)x(n k) k i e (i) (n) x(n i) ˆx(n i) = c k (n)x(n k) ˆx(n i) Change in Notation c k(n)x(n k) k i e (i) (n) c k (n)x(n k) Note that this is inconsistent with the notation used earlier, ŷ o (n) = M 1 h o (k)x(n k) = M 1 c k+1x(n k) Also the sums have M +1terms in them, rather than M terms as before Presumably motivated by a simple expression for the error e (i) (n) J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 3 J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 4

2 ˆx(n i) Types of Prediction c k(n)x(n k) k i e (i) (n) c k (n)x(n k) Forward Linear Prediction: i = Backward Linear Prediction: i = M Misnomer, but terminology is rooted in the literature Symmetric Linear Smoother: i = M/2 Linear Prediction Notation and Partitions x(n) [ x(n) x(n 1) x(n M +1) ] T x(n) [ x(n) x(n 1) x(n M) ] T = [ x(n) x T (n 1) ] T = [ x T (n) x(n M) ] T R(n) =E[x(n)x H (n)] R(n) =E[ x(n) x H (n)] Forward and backward linear prediction use specific partitions of the extended autocorrelation matrix [ Px (n) r R(n) f H(n) ] [ ] R(n) rb (n) R(n) r f (n) R(n 1) rb H(n) P x(n M) r f (n) =E[x(n 1)x (n)] r b (n) =E[x(n)x (n M)] J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 Forward and Backward Linear Prediction Estimator and Error ˆx f (n) = a k(n)x(n k) = a H (n)x(n 1) k=1 M 1 ˆx b (n) = b k(n)x(n k) e f = x(n)+ e b = M 1 = b H (n)x(n) a k(n)x(n k) = x(n)+a H (n)x(n 1) k=1 b k(n)x(n k)+x(n M) = b H (n)x(n)+x(n M) Again, new notation compared to the FIR linear estimation case I use subscripts for the f instead of superscripts like the text Forward and Backward Linear Prediction Solution Solution is the same as before, but watch the minus signs R(n 1)a o (n) = r f (n) R(n)b o (n) = r b (n) P f,o (n) =P x (n)+rf H (n)a o (n) P b,o (n) =P x (n M)+rb H (n)b o (n) ˆx f (n) = a k(n)x(n k) k=1 = a H (n)x(n 1) M 1 ˆx b (n) = b k(n)x(n k) = b H (n)x(n) J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 8

3 Stationary Case: Autocorrelation Matrix When the process is stationary, something surprising happens! R (M+1) (M+1) = 2 4 r x () r x (1) r x (M 1) r x (M) rx(1) r x () r x (M 2) r x (M 1) rx(m 1) rx(m 2) r x () r x (1) rx(m) rx(m 1) rx(1) r x () r T r x (1) r x (2) r x (M) Λ M 1 R(n) = [ Px (n) rf H(n) ] [ ] R(n) rb (n) = r f (n) R(n 1) rb H(n) P x(n M) 3 R (M+1) (M+1) = Stationary Case: Cross-correlation Vector 2 4 r x () r x (1) r x (M) rx(1) r x () r x (M 1) rx(m) rx(m 1) r x () [ ] rx () r R = f H r f R [ ] R rb R = rb H r x () 3 [ ] rx () r = T r R [ ] R Jr = r H J r x () r M r x (1) r x (2) r x (M) 3 Clearly, R(n) =R(n 1) P x () = P x (n M) =r x () r f =E[x(n 1)x (n)] r b =E[x(n)x (n M)] where J is the exchange matrix = r = Jr J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 9 J McNames Portland State University ECE 39/39 Linear Prediction Ver J 1 1 Exchange Matrix J H J = JJ H = I Counterpart to the identity matrix When multiplied on the left, flips a vector upside down When multiplied on the right, flips a vector sideways Don t do this in MATLAB many wasted multiplications by zeros See fliplr and flipud Forward/Backward Prediction Relationship Rb o = r b = Jr Ra o = r f = r JRa o = Jr JR a o = Jr = r b JR = RJ R(Ja o)= r b b o = Ja o The BLP parameter vector is the flipped and conjugated FLP parameter vector! Useful for estimation: can solve for both and combine them to reduce variance Further the prediction errors are the same! P f,o = P b,o = r() + r H a o = r() + r H Jb o J McNames Portland State University ECE 39/39 Linear Prediction Ver J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 12

4 Example 1: MA Process Create a synthetic MA process in MATLAB Plot the pole-zero and transfer function of the system Plot the MMSE versus the point being estimated, for M =2 ˆx(n i) c k(n)x(n k) k i MNMSE Example 1: MMSE Versus Prediction Index i Minimum NMSE Estimated MNMSE Prediction Index (i, Samples) J McNames Portland State University ECE 39/39 Linear Prediction Ver J McNames Portland State University ECE 39/39 Linear Prediction Ver Example 1: Prediction Example Example 1: Prediction Example M:2 i: NMSE:388 x(n) ˆx(n +) M:2 i:13 NMSE:18 x(n) ˆx(n +13) Signal + Estimate (scaled) Signal + Estimate (scaled) Sample Time (n) Sample Time (n) J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 1 J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 1

5 Signal + Estimate (scaled) Example 1: Prediction Example M:2 i:2 NMSE:388 x(n) ˆx(n +2) Sample Time (n) Example 1: MATLAB Code clear all; close all; N = ; % Number of samples M = 2; % Size of filter b = poly([99 98*j -98*j 98*exp(j*8*pi) 98*exp(-j*8*pi)]); nz = length(b)-1; % Number of zeros Mmax = M+1; % Maximum value to consider % Calculate the Auto- and Cross-Correlation rx = conv(b,fliplr(b)); % Quick calculation of rx k = -nz:nz; rx = rx(nz+1:end); % Trim off negative lags ryx = rx(2:end); % Cross-correlation one-step ahead % Generate Example w = randn(n,1); x = filter(b,1,w); % Build extended R Re = zeros(m+1,m+1); for c1=1:m+1, for c2=1:m+1, id = abs(c1-c2); if id<=nz, Re(c1,c2) = rx(id+1); % Locations of zeros J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 1 J McNames Portland State University ECE 39/39 Linear Prediction Ver Po = zeros(m+1,1); X = zeros(n,m+1); Poh = zeros(m+1,1); for id=:m, R = [Re(1:id,1:id),Re(1:id,id+2:end); Re(id+2:end,1:id),Re(id+2:end,id+2:end)]; d = [Re(1:id,id+1);Re(id+2:end,id+1)]; % Extract i th column sans the ith row Px = Re(id+1,id+1); co = -inv(r)*d; Po(id+1) = Px + d *co; X(:,id+1) = filter(-[co(1:id);;co(id+1:end)],1,x); k = 1:N-(id+1); Poh(id+1) = mean((x(k)-x(k+id,id+1))^2); % Plot MMSE Versus Order h1 = plot3(:m,poh/var(x),-1*ones(m+1,1), ro ); set(h1, MarkerFaceColor, r ); set(h1, MarkerSize,); view(,9); hold on; h2 = stem(:m,po/px, k ); set(h2, MarkerFaceColor, k ); set(h2, MarkerSize,4); hold off; xlabel( Prediction Index (i, Samples) ); ylabel( MNMSE ); xlim([- M+]); ylim([ 1]); AxisLines; legend([h1;h2], Minimum NMSE Estimated, MNMSE ); print( MAExampleMNMSEvi, -depsc ); % Plot Estimates id = [,round(m/2),m]; for c1=1:length(id), k = 1:N-(id(c1)+1); xh = X(k+id(c1),id(c1)+1); FigureSet(2, LTX ); h = plot(k,x(k), b,k,xh(k), g ); set(h, LineWidth,8); set(h, Marker, ); set(h, MarkerSize,8); xlim([ 1]); ylim(std(x)*[-3 3]); AxisLines; xlabel( Sample Time (n) ); ylabel( Signal + Estimate (scaled) ); set(get(gca, Title ), Interpreter, LaTeX ); title(sprintf( M:%d i:%d $\\widehat{\\mathrm{nmse}}$:%3f,m,id(c1),mean((x(k)-xh)^2)/var(x(k)))); hl = legend(h, $x(n)$,sprintf( $\\hat{x}(n+%d)$,id(c1))); set(hl, Interpreter, LaTeX ) print(sprintf( MAExamplePlot%2d,id(c1)), -depsc ); J McNames Portland State University ECE 39/39 Linear Prediction Ver J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 2

6 Selected Properties If x(n) is stationary, The linear smoother has linear phase The forward prediction error filter (PEF) is minimum phase The backward PEF is maximum phase The forward and backward prediction errors can be expressed as P f,o (n) = det R(n) det R(n 1) P b,o (n) = Other properties and proofs are given in the booik det R(n) det R(n) Example 2: Financial Time Series Forecast Try predicting a tremor signal acquired from an accelerometer attached to the wrist of a patient with Parkinson s disease J McNames Portland State University ECE 39/39 Linear Prediction Ver J McNames Portland State University ECE 39/39 Linear Prediction Ver Example 2: Data Details The files tremorparkgz and tremorphysgz contain measurements of the acceleration of the outstretched hand which is supported and fixed at the wrist The units are arbitrary The sampling frequency is 3 Hz tremorphysgz shows the tremor of a healthy person, tremorparkgz that of a person suffering from Parkinson s disease The real amplitude of the latter is of course much larger than that of the former For further information see: or mail to:jeti@fdmuni-freiburgde Jens Timmer Acceleration (scaled) N: Time (s) J McNames Portland State University ECE 39/39 Linear Prediction Ver J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 24

7 1 Autocorrelation 12 Partial Autocorrelation 1 8 ρ ρ Lag (s) Lag (s) J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 2 J McNames Portland State University ECE 39/39 Linear Prediction Ver x 1 Blackman-Tukey Estimated PSD PSD (scaled) 8 4 Frequency (Hz) Frequency (Hz) Signal Time (s) J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 2 J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 28

8 3 2 NMSE:42 P:1 M:2 Signal Predicted 1 8 P:1 Acceleration (scaled) NMSE Time (s) M (samples) J McNames Portland State University ECE 39/39 Linear Prediction Ver J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 3 NMSE M: P(s) Example 2: MATLAB Code clear; close all; % Load the Data x = load( R:/Tremor/Parkinsdat ); x = x - mean(x); fs = 3; % Sample rate (Hz) nx = length(x); % Length of data k = 1:nx; % Sample index t = (k-)/fs; % Sample times % Plot the Data h = plot(t(1:1:end),x(1:1:end), r ); set(h, LineWidth,8); xlabel( Time (s) ); ylabel( Acceleration (scaled) ); title(sprintf( N:%d,nx)); xlim([t(1) t(end)]); ylim(prctile(x,[ 99])); print( RESignal, -depsc ); % Plot the Autocorrelation Autocorrelation(x,fs,); J McNames Portland State University ECE 39/39 Linear Prediction Ver J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 32

9 print( REAutocorrelation, -depsc ); % Plot the Partial Autocorrelation PartialAutocorrelation(x,fs,); print( REPartialAutocorrelation, -depsc ); % Plot the Power Spectral Density BlackmanTukey(x,fs,1); print( REBlackmanTukey, -depsc ); % Plot the Spectrogram NonparametricSpectrogram(decimate(x,),fs/,2); print( RESpectrogram, -depsc ); % Calculate the Auto- and Cross-Correlation rx = Autocorrelation(x,fs,2); m = 2; % Filter length p = 1; % Number of steps ahead to predict R = zeros(m,m); for c2=1:m, R(c1,c2) = rx(abs(c1-c2)+1); d = zeros(m,1); d(c1) = rx(c1+p); co = inv(r)*d; xh = filter(co,1,[zeros(p,1);x]); xh = xh(1:nx); NMSE = mean((x-xh)^2)/mean(x^2); % Plot Segment of Signal and Predicted h = plot(t,x, r,t,xh, g ); set(h(1), LineWidth,8); set(h(2), LineWidth,12); xlabel( Time (s) ); ylabel( Acceleration (scaled) ); title(sprintf( NMSE:%3f P:%d M:%d,NMSE,p,m)); legend(h, Signal, Predicted ); xlim([t(1) ]); ylim(prctile(x,[ 99])); print( RESignalPredicted, -depsc ); % Sweep M for P=1 J McNames Portland State University ECE 39/39 Linear Prediction Ver J McNames Portland State University ECE 39/39 Linear Prediction Ver p = 1; % Number of steps ahead to predict M = 1:1; nm = length(m); NMSE = zeros(nm,1); for c=1:nm, m = M(c); % Filter length R = zeros(m,m); for c2=1:m, R(c1,c2) = rx(abs(c1-c2)+1); d = zeros(m,1); d(c1) = rx(c1+p); co = inv(r)*d; xh = filter(co,1,[zeros(p,1);x]); xh = xh(1:nx); NMSE(c) = mean((x-xh)^2)/mean(x^2); h = plot(m,nmse, b ); set(h(1), LineWidth,8); xlabel( M (samples) ); ylabel( NMSE ); title(sprintf( P:%d,p)); xlim([ M(end)+]); ylim([ 1]); print( RESweepM, -depsc ); % Sweep P for P=1 P = 1:3; % Number of steps ahead to predict m = 2; np = length(p); NMSE = zeros(np,1); R = zeros(m,m); for c2=1:m, R(c1,c2) = rx(abs(c1-c2)+1); for c=1:np, p = P(c); % Filter length d = zeros(m,1); d(c1) = rx(c1+p); co = inv(r)*d; xh = filter(co,1,[zeros(p,1);x]); xh = xh(1:nx); NMSE(c) = mean((x-xh)^2)/mean(x^2); h = plot(p/fs,nmse, b ); set(h, LineWidth,8); set(h, Marker, ); set(h, MarkerSize,); hold on; hb = plot([ P(end)/fs],[1 1], r: ); hold off; J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 3 J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 3

10 xlabel( P (s) ); ylabel( NMSE ); title(sprintf( M:%d,m)); xlim([ P(end)/fs]); ylim([ 1]); print( RESweepP, -depsc ); J McNames Portland State University ECE 39/39 Linear Prediction Ver 12 3